Initial results and mixed directions for Research Methods Tutor 1

RMT (Research Methods Tutor) is a dialog-based tutoring system currently being used in conjunction with university-level courses on research methods in psychology. RMT has a web-based interface and uses a talking head to to present its dialog acts to the student. The student types in unconstrained natural language responses. Although RMT currently focuses on natural language-based interaction with students, the use of the talking head brings in other modalities of interaction which must be integrated with the textual “message”. This paper describes the basic architecture and approach of the RMT system, and its evaluation in the context of recent research methods classes. Our experiments compared the agent-based tutor to a text-only version of the system, and compared tutoring to a computer-aided instruction control. Due to technical difficulties with the agent software and some compliance issues with the students, we got no significant results that validated the usefulness of the system as a learning aid. We did however see consistent trends of additional learning in conjunction with the use of the tutor that warrant further investigation. This paper concludes by describing our current efforts for integrating graphical visual aids with dialog-based tutoring.